Self-complementary plane partitions by Proctor's minuscule method 
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A method of Proctor M] realizes the set of arbitrary plane partitions in a box and the set of symmetric plane 
partitions as bases of linear representations of Lie groups. We extend this method by realizing transposition and 
complementation of plane partitions as natural linear transformations of the representations, thereby enumerat- 
ing symmetric plane partitions, self-complementary plane partitions, and transpose-complement plane partitions 
in a new way. 



While investigating minuscule representations of Lie alge- 
bras, Robert Proctor [Q] developed one of the simplest meth- 
ods for enumerating symmetry classes of plane partitions. 
The method yields an enumeration of arbitrary plane parti- 
tions, using the representation theory of SL(n), and symmetric 
plane partitions, using the (projective) representation theory 
of SO(2n +1). (Proctor also provided solutions to two previ- 
ously open symmetry classes, namely TCPP's and SSCPP's 
in the notation of Stembridge, in a related representation- 
theoretic investigation.) 

The author and John Stembridge have independently dis- 
covered a way to extend this method to enumerate self- 
complementary plane partitions (first enumerated by Stanley 
|g]). In this paper, we will present a brief proof of this enumer- 
ation; Stembridge [[7]] gives a longer exposition and another 
generalization of the new proof. For both of us, the proof was 
inspired by, and partially explains, Stembridge's q = — 1 phe- 
nomenon OS), which in this case is the observation that the 
number of self-complementary plane partitions is the number 
of plane partitions with an even number of cubes minus the 
number with an odd number of cubes. 

In addition, the author has found a new proof of the enu- 
meration of symmetric plane partitions which extends to an 
enumeration of TCPP's, while Stembridge's analysis also enu- 
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Table I: Proctor's minuscule method and its extensions 



merates SSCPP's and self-complementary chains of order 
ideals in other minuscule posets. Unfortunately, symmetry 
classes involving cyclic symmetry have no known treatment 
by the minuscule method. Table | gives a summary of which 
enumerations are given in which paper. 



1. CONVENTIONS 
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Figure 1 : A plane partition in a box 

A plane partition in an a x b x c box is a collection of unit 
cubes in the rectangular solid [Q,a] x [0,b] x [0,c] in three 
dimensions which is stable under gravitational attraction to- 
wards the origin; Figure [y shows an example of a plane parti- 
tion. 

The two simplest symmetry operations on the set of plane 
partitions are transposition T, which consists of switching two 
coordinate axes; and rotation p, which consists of cyclically 
permuting all three axes. There is also a third operation, com- 
plementation K, which consists of taking all cubes which are 
in the box but not in the diagram of the partition and reversing 
all three coordinates of the box. 

A partition in an a x b rectangle is the analogue of a plane 
partition in two dimensions, as shown in Figure |[ The op- 
erations of complementation and transposition act on parti- 
tions. The set of partitions has a partial ordering given by 
set-theoretic inclusion, and it will be convenient to think of 
a plane partition as an ordered chain of c partitions. (Adja- 
cent terms may be equal in such a chain.) Transposition of 
each partition in such a chain is equivalent to transposition of 
the plane partition, and the same is true of complementation 
provided the chain is also reversed. 



Figure 2: A partition in a rectangle 



A partition P in an a x b rectangle can also be viewed as 



a a binary sequence, i.e., a function b P from {1, 



b} to 



These elements are not a basis of S c (A a V ), but merely a lin- 
early independent set. 

We view V, A a V, and S c (A a V) as linear representations of 
SL(a + b) = SL(V). The representation S c (A a V) has a unique 
highest weight vector with weight cX a . Here X n is the nth fun- 
damental weight, the weight corresponding to the nth vertex 
from the end of the Dynkin diagram of SL(a + b), which is a 
chain of length a + b — 1 . Therefore, the irreducible represen- 
tation V(cX a ) appears as a subrepresentation with multiplicity 
1, and there is an equivariant projection 

7t:S c (A a V)->V(cX a ). 

(Those not familiar with the representation theory of simple 
Lie algebras should refer to Serre [ph or Bourbaki [fit].) 
We now invoke the following fundamental result: 



{0, 1} whose values sum to a. Specifically, the boundary of 
P is a path of a + b steps from the upper left corner to the 
lower right corner, and we let b p (n) = 1 if the n'th step goes 
down and b P (n)=0if it goes to the right. Transposing or Theorem 2.2. The vectors x T = %{p T ) ranging over all plane 



complementing a partitions has a simple effect on its binary 
sequence: 



K(p) {n) = b p (a + b+l-n) 
l-b P (n). 



b , n \(n 



Finally, if T is a plane partition (or a partition), \T\ is the 
number of cubes (squares) in T, and the g-weight of T is de- 
fined to be q\ T \. 



2. ARBITRARY PLANE PARTITIONS 



This section is a review of the following result: 

Theorem 2.1. The set of plane partitions in an a x b x c box 
can be regarded as a basis of an irreducible representation of 
SL(a + b). 

This result is essentially due to Young, Schur, and Weyl, 
but it was first noted in the context of plane partitions by Stan- 
ley and Proctor, and Proctor generalized it to symmetric plane 
partitions and SO (2a + 1 ). 

Consider a vector space V over C (some other choices of 
the ground field work also) of dimension a + b and with basis 
jcj , . . . ,x a+b . The vector space A a V of alternating forms over 
V of degree a has a basis indexed by partitions in an a x b 
box. Namely, the basis element corresponding to P is a wedge 
product formed from the binary sequence of P: 

a+b 
n=\ 

Consider the vector space S c (A a V) of symmetric forms 
over A"V of degree c, i.e., homogeneous polynomials of de- 
gree c in the x P 's, treated as independent variables. If a plane 
partition T is viewed as a chain of partitions P^ > P 2 > . . . > 
P c , then we can define an element p T of S C (A"V) by 



Pt 



IP 



partitions T in an ax bx c box, are a basis for the irreducible 
representation V(cX a ) ofSh(a + b). 

The proof follows from the theory of semistandard 
tableaux, originally due to Young, as applied by Weyl to Lie 
groups [§], 

Using this result, the Weyl dimension formula gives us the 
number of plane partitions in a box. In the notation of Propp, 
the number is: 



N(a,b,c) 



H(a + b + c)H(a)H(b)H(c) 
H(a + b)H(a + c)H(b + c) 



Here H(n) — 1 !2!3! ... (n — 1)! is the hyperfactorial function. 
More generally, the Weyl g-dimension formula gives the trace 
of the element 



D„ 



q 



(l-a-b)/2 



( 1 
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a+b-l 



\ 



in SL(a +b) in its action on V(cX a ) (or any other irreducible 
representation), which is important to us because of the fol- 
lowing lemma: 

Lemma 2.3. The the action of D q <G SL(a + b) on V(cX a ) 
is diagonal in the basis {x T }, and the eigenvalue of x T is 



q 



T\-abc/2 



Proof We will work our way up from x n for an integer n to x P 
for a partition P to p T and then to x T . The element D q is, by 
definition, diagonal in its action on V, and the eigenvalue of 
x n is 4»-(«+*+l)/2. 

Since x p is a (wedge) product of x„'s, it is also an eigenvec- 
tor of D q , and its eigenvalue is the product of the eigenvalues. 
If Pq is the empty partition, then 



= Xj Ax 2 A . 



and the eigenvalue is therefore q "{a+i)/2-a(a+b+l)/2 _ q -ab/2 
Consider the effect of an elementary move on P which con- 
sists of adding a single square to it. This is equivalent to re- 
placing some factor x„ inx^by x n+l , which results in an extra 
factor of q. Therefore, the eigenvalue is q\ p \^ ab l 2 in the gen- 
eral case. 

Since p T is a (symmetric) product of c different x P 's, it is 
also an eigenvector of D q . Its eigenvalue is the product of the 
eigenvalues of its factors, namely q\ T \^ abc l 2 . 

Finally, D q commutes with the projection % (because n is 
equivariant under the action of all of SL(a + b)). Thus, if p T 
is an eigenvector with a certain eigenvalue, x T = 7i(p T ) must 
be an eigenvector with the same eigenvalue. □ 



The logic of the proof of Lemma 2.3 will be repeated sev 



eral more times in the rest of the paper with successively 
shorter explanations. 

The ^-dimension formula yields the trace of D q , which by 
the lemma is q- abc / 2 times the total g-weight of all plane par- 
titions. The formula for the g- weight is: 



N(a,b,c) q 



H{a + b + c) q H{a) q H(b) q H(c) q 
H{a + b) q H(a + c) q H(b + c) q 



where 



ff(n) 9 = (l) 9 !(2),!...(n-l),! 
is the g-hyperfactorial function, 

(*),! = (l) q (2) q ...(n) q 
is the g-factorial function, and 



(n) g = l+q + q 2 +...+q" ' 



is a ^-integer. 



3. SELF-COMPLEMENTARY PLANE PARTITIONS 

Theorem 3.1 (Stembridge, K). If the plane partitions in an 
a x b x c box are a basis for a vector space, the operation 
of complementation is linearly conjugate to the operation of 
negating partitions with an odd number of cubes. 

We consider the effect of the group element of the form: 



/0 1 \ 
10 
10 
10 

V 1 0/ 



K 



■a+b- 1 



in SL(a + b) on the jc p 's and the jc r 's. By definition, it sends 
x„ to i a x +b+l _ n , and its effect on a partition is to reverse 
its binary sequence, which is the complementation operation. 
Specifically: 



K(x P ) = i a{a+b - x \-l) a{a -^l 2 x, p) = i ah x 



The factor of (— l)"( a_1 )/ 2 comes from a reversal of the fac- 
tors in x p , which is an antisymmetric product. Since p T is a 
symmetric product of x P 's, the effect of K is: 



K(p 7 



1 Pk(T) 



Finally, since K commutes with K, it has the same effect on 
the jc r 's. Therefore the trace of K in V{cX a ) is, up to a factor 
of i abc , the number of plane partitions fixed by K. V(bX a ). 

But K is conjugate to D { and therefore has the same trace. 
(Note that the notation D q is an abuse because the matrix is 



really a function of q 1 ' 2 . 
conclusion, 



To obtain D _,, we set q 1 ' 2 — i.) In 



Corollary 3.2 (Stanley). 

N K (a,b,c) =N(a,b,c)_ l 

This equation is easily equivalent to the formula given by 
Stanley [§. 

4. SYMMETRIC PLANE PARTITIONS 

Theorem 4.1. // the plane partitions in an a x a x c box 
are interpreted as a basis for an irreducible representation of 
SL(2a) as in section pi transposition is induced by the Hodge 
star operator relative to a symplectic inner product. 

Proof. We proceed with the notation of the section H, taking 
a = b. For any non-degenerate bilinear form B onV, the group 
SL(V) possesses an outer automorphism o„:A^ A~ T (in- 
verse transpose), where the transpose A i— > A is taken relative 
to the form B. (Usually, B is either symmetric or antisymmet- 
ric, but it need not be either in the definition of a B .) Explicitly, 
B induces an isomorphism a B : V — ► V*, where V* is the dual 
vector space, given by 

a B (y)=B(-,v). 

Then O b {A) is the composition 

0t B A (X B , 

where A* is the adjoint of A. Even more explicitly, if A and B 
are given by matrices M A and M B , 



M 



ffjW 



-M B l M A T M B , 



kP) 



where M~ T is the usual inverse transpose of the matrix M. We 
consider the group Z/2 « a SL(V), which is the same group 
for all choices of B because a B differs from a B , by an inner 
automorphism of SL(V). 

The automorphism a B can be given a compatible linear ac- 
tion on A"V, called the Hodge star operator, as follows: First, 
we pick a specific volume form on V, i. e. an isomorphism 
between A 2a V and C, relative to which B has determinant 1 . 
This identification induces a bilinear form on A"V given by 
the wedge product. There is a second bilinear form induced 



by B (and denoted by the same letter), because A a V C V® a 
and V m has a bilinear form which is just a multilinear exten- 



sion of S on V. More explicitly, if v 1? . 
are two sequences of vectors, 



,v a and w, ,. 



,Wa 



B{v 1 A...Av a ,w 1 A...Aw a ) = £ (- 1 ) ff f[ 2? ( v '» l V» 



aes a 



n=\ 



in)' 



(1) 

Both bilinear operations can be interpreted as functions from 
A"V to its dual, and the Hodge star operator is either one com- 
posed with the inverse of the other. More explicitly, the Hodge 
star is defined by the equation 

B(v,*fi>) = vAffl 

for v, CO <E A a V. If it is assigned to a B , A a V becomes a repre- 
sentation of Z/2 K a SL(V). 

With respect to the given basis for V, we define volume in 
the standard way, and we define B by a matrix of the form: 

/ 1 \ 

0-10 

10 

0-1000 

10 
\-10 j 

In general, B is defined by the equations 



oyXniXfcj 



With this choice of B, 



(-1)" k = 2a+l-n 
otherwise 



(2) 



B(x, 



p,^p! 



= 



unless P' = k{P), and 

B(x P ,x K( p ) ) = {-l)\ p \, 

by multilinearity. The sign in this equation merits some expla- 
nation. A factor of (— iy p \ +a ("~ ] -)/ 2 comes from multiplying 
a factors of equation 0, while another factor of (— l) a ( a_1 )/ 2 
appears because in the sole non-zero term in equation [y, o is 
the reversing permutation. 
On the other hand, 



JC p / \ JC p/ 







unless b P (n) = 1 —b p ,(n), because otherwise the wedge prod- 
uct has repeated linear factors. This happens precisely when 
P' = kt(P). In this case: 



x p Ax ki(p) = (-l)' P '*iA...Ax 



2a- 



The sign comes from permuting the wedge factors. Compos- 
ing the wedge product (as a scalar-valued bilinear function) 
with B, we obtain 



which establishes that the Hodge star operation as the lin- 
ear extension of transposition of partitions. We can extend 
the Hodge star multilinearly to the polynomial space S c (A a V) 
over a-forms, and it immediately follows that 

*PT=Pr( T) - ( 3 ) 

We would like to infer the conclusion of the theorem, 



*x T 



\(T) 



from equation |J To do so, we need to know that the Hodge 
star operator commutes with the projection % : S C (A"V) — > 
V(cX a ). In general, given a representation R of a semidirect 
product Z/2 K a G, with G reductive, either a, acting on G, 
fixes the character of a G-irreducible summand V of R, in 
which case a, acting on R, leaves V invariant; or a does not 
fix the character of V, in which case (7(V) is a disjoint G- 
irreducible summand of R. In our case, V(cX a ) appears with 
multiplicity 1, and % is characterized as the projection whose 
image is V (cX a ) and whose kernel is the direct sum of all other 
SL(V)-irreducible summands. With the given choice of B, the 
automorphism a B is induced by reversing the Dynkin diagram 
of SL(V), which fixes its middle fundamental weight X a and 
therefore also fixes cX a and the entire character of V(cX a ). 
Therefore both the kernel and image of % are invariant under 
the Hodge star action of a B , which implies that % and * com- 
mute. □ 

As stated in the proof, a B is a Dynkin diagram automor- 
phism. The character theory of semi-direct products arising 
from Dynkin diagram automorphisms is described by Neil 
Chriss [HJ, who explained to the author that although this the- 
ory is known to several representation theorists, it may not 
have been previously published. The group Z/2 k a SL(2a) 
has two components. The character of a representation on 
the identity component is just the usual character of SL(2a). 
The character on the o B component, when non-zero, equals 
the character of an associated representation of the dual Lie 
group, in this case SO(2a + 1), to the subgroup fixed by the 
outer automorphism, in this case Sp(2a). The representa- 
tion associated to VsL(2al( c '^ 2 ) ' s tne P ro J ectrv e representation 



V, 



SO(2a+l 



«, 



-TP) 



(cX a ), where X a is now the weight corresponding to 
the short root of B a , the root system of SO(2a + 1). In par- 
ticular, the trace of a B is the dimension of V , so / 2a+1 \(cA a ), as 
given by the Weyl dimension formula, and the trace of CJ B D q is 
the g-dimension, as given by the Weyl g-dimension formula. 
From the construction of these linear operators, their traces 
are N T (a,a,c), the number of T-invariant plane partitions, and 
their total ^-weight, N T (a,a,c) i! , times q~"~ c ' 2 , respectively. 

By analogy with the results described in section 0, Proc- 
tor [Q| also realizes T-invariant plane partitions as a basis for 
the same projective representation VJbq/^ , ■n(cAa), an d me <?~ 
dimension again equals the trace of the linear transformation 
which multiplies each plane partition by its ^-weight times 
q-a c/2 5 mce tne resu it of this section arrives at the same Lie 
group character, if not an explicit representation, it provides 
another proof which is similar to but not the same as Proctor's 



proof of the product formulas for N T (a,a,c) and N T (a,a,c) q . 
Converting the Weyl formulas by an extension of Propp's no- 
tation, they are 



N T {a,a,c) q 



Here 



H 2 (2a + b) 2 H(a) z (2a + b-l) q U 
H(a + b) 2 H 2 (b+l) 2 (b-l) q \\(2a-l) q \\ 



H 2 (n) = {n-2) q l(n-4) q l{n-6) q l... 

is a staggered g-hyperfactorial, and 

n q \\ = (n) q (n-2) q (n-4) q ... 

is a staggered g-factorial. (The formula was first proved by 
George Andrews.) 

5. TRANSPOSE-COMPLEMENT PLANE PARTITIONS 

In this section, we combine the results of the previous two 
sections to enumerate KT-invariant plane partitions. In the 
representation V(cX a ) of Z/2 K a SL(2a), which has plane 
partitions in an a x a x c box as a basis, the group element 
a B acts as the linear extension of t, while the group ele- 

9 

ment K acts as i" c times the linear extension of k. Therefore 
their product acts as i" c times kt. If a B K were conjugate to 
o B D j , it would imply by the results of the last two sections 
that 

N TK (a,a,c) — A^ T (a,a,c)_j, 

which easily implies the usual formula due to Proctor and 
mentioned by Stanley [|[. It is easy to check that a group 
element with a matrix of the form 



1 
V2 



( 1 l\ 
1-10 
110 

V-i ooi/ 



conjugates K to D_ i and commutes with a B . (The general 
expression for this element is -X=(I + D_ l K~ l ), or its matrix 



is -4= (I+M B ).) Therefore o B K and a B D _j are also conjugate. 
The conclusion is 



Theorem 5.1. 



N KT (a,a,c) —N T (a,a,c)_, 



SYMMETRIC, SELF-COMPLEMENTARY PLANE 
PARTITIONS 



In this paper, we have constructed a linear representation 



Vc 



SL(o+i) 



(cX a ) with a basis indexed by plane partitions. We 
have recognized the linear extension of complementation K as 
part of the group action on this representation. When a = b, 
we have also recognized the linear extension of transposi- 
tion T as part of the group action of a semidirect product of 
SL(a + b). The traces of these linear transformations tell us 
the number of elements fixed by K or T or kt. The author 
briefly thought that this construction would lead to an enu- 
meration of plane partitions fixed by both k and T. this is 
a naive hope. Consider a permutation representation of the 
Klein group generated by T and K with one free orbit and two 
fixed points, and consider a second permutation representa- 
tion with three orbits with two points, with stabilizers t, k, 
and kt. These two representations are different, but the linear 
representations they induce are isomorphic. 

Thus, in the absence of extra structure, the linear represen- 
tation V SL(2a JcA a ) gives no information about K, T-invariant 
plane partitions. However, Stembridge enumerates k,t- 
invariant plane partitions by expressing K as an involution on 
the representation V SQ , 2a+1 , (cX a ). The author conjectures that 
latter representation can be realized as a natural vector sub- 
space of the former one. 
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